1. Field of the Invention
This invention relates to a picture information conversion method and apparatus for converting the bit rate of the compressed picture information.
2. Description of Related Art
Recently, such an apparatus is in widespread use in which the picture information is handled as digital data, which digital data is compressed by orthogonal transform and motion compensation exploiting redundancy inherent in the picture information to effectuate or transmission to a network media, such as satellite broadcast or cable television, or recording on a storage medium, such as a magnetic disc. In such apparatus, the MPEG-2 (Moving Picture Experts Group-2) employing the discrete cosine transform is used in general as the picture compression system.
Recently, normalization of the digital television broadcasting, employing this picture compression system, exemplified by MPEG-2, is underway. Among the standards of the digital television broadcasting, there are a standard associated with a picture of standard resolution, such as a picture having 576 effective lines in the vertical direction, and a standard associated with a picture of high resolution, such as a picture having 1152 effective lines in the horizontal direction.
Meanwhile, the picture information of the high resolution picture is voluminous, such that, even on compression by the encoding system, such as MPEG-2, an extremely large amount of codes (bit rate) is required in order to achieve a sufficient picture quality. For example, in the case of interlaced scanning at 30 Hz for a picture frame of 1920 by 1080 pixels, the bit rate is of the order of 18 to 22 Mbps or higher.
So, if this high resolution picture is to be transmitted by satellite broadcast or over a network media such as cable television, the bit rate needs to be diminished further in meeting with the bandwidth of the transmission channel. Similarly, in recording this high resolution picture on a storage media, such as an optical disc or a magneto-optical disc, the bit rate needs to be diminished further in meeting with the recording capacity of the storage media. The necessity of diminishing the bit rate is thought to arise not only for a high resolution picture but also for a standard resolution picture, such as a picture obtained on interlaced scanning at 30 Hz for a picture frame of 720 by 480 pixels.
Among the means for combatting this problem, there are hierarchical encoding (scalability) or picture information conversion (transcoding). In MPEG-2, SNR scalability is normalized for the former. Using this, the picture compression information (bitstream) of high SNR and the picture compression information (bitstream) of high SNR are hierarchically encoded. However, even though a pre-set value such as bandwidth or recording capacity needs to be known at the time point of encoding in order to effect the hierarchical encoding, this value is not known in many cases in the actual system.
Referring to the drawings, a conventional picture information conversion device 100 is explained. This conventional picture information conversion device 100 is shown in FIG. 1.
The conventional picture information conversion device 100 includes a picture information decoding device 101 and a picture information encoding device 102, as shown in FIG. 1. This conventional picture information conversion device 100 is generally a device for reducing the bitrate of the compressed picture information (bitstream) and is adapted for supplying the picture information from the picture information decoding device 101 to the picture information encoding device 102.
First, the picture information decoding device 101 of the picture information conversion device 100 is fed with the high bitrate compressed picture information. The picture information decoding device 101 once decodes the high bitrate compressed picture information completely to output baseband video data.
The picture information encoding device 102 is fed at the outset with a target bitrate lower than the code volume (high bitrate) of the input compressed picture information, and executes the encoding based on this target bitrate. That is, the picture information encoding device 102 re-encodes the baseband video data, obtained as an output of the picture information decoding device 101, to output the compressed low bitrate picture information.
Recently, in this conventional picture information conversion device 100, researches towards simplifying the hardware architecture and improving the image quality are underway. In particular, the development of an algorithm is going on briskly in connection with optimisation of re-quantization and controlling of the bitrate.
In the conventional picture information conversion device 100, the large number of the algorithms, developed for optimum quantization and optimum bitrate control, are not exploited effectively. These algorithms improve the SNR (average picture quality) of the conventional picture information conversion device 100 merely to a limited extent.
On the other hand, bit allocation to each picture in the conventional picture information conversion device 100 is based on the Test Model 5 (ISO/IEC JTC1/SC29/WG11 No.400), referred to below as TM5. In this TM5, the target bitrate allocated to each residual picture allocated to each residual picture is calculated using a calculated value of the complexity of each residual picture.
FIG. 7 shows transition of the frame-based complexity obtained on calculating the complexity of each of the 60 frames by a picture sequence sprinkler. In this picture sequence sprinkler, each GOP is made up of 15 pictures of I, B, B, P, B, B, P, B, B, P, B, B, P, B, B.
Moreover, in the conventional picture information conversion device 100, the bitrate of the input compressed picture information (bitstream) is used as the target bitrate.
Assume that the i""th picture in the GOP, made up e.g., of 15 pictures, is being processed. If the conventional picture information conversion device 100 is not exploiting the feed-forward technique, this conventional picture information conversion device 100 is unable to recognize a calculated value of the complexity of the residual picture extracted from the i""th residual picture in the GOP. In the TM5 in such case, the conventional picture information conversion device 100 uses calculated value of the complexity generated from the latest encoded picture. Meanwhile, the feed-forward technique herein means the technique of determining an manipulated variable based on the command value of a manipulated variable of a controlled object or on the information such as disturbance.
The complexity value of a picture for each frame is increased rightwards up monotonously from a frame 1 to a frame 61, as shown in FIG. 7. However, in actuality, a target bitrate smaller than a command bitrate to be allocated is allocated to each residual picture. If, in the conventional picture information conversion device 100, such inappropriate bit allocation is used, underflow occurs readily to deteriorate the picture quality, because the bitrate cannot be curtailed in case the command bitrate to be allocated to each residual picture is insufficient.
Moreover, if, in the conventional picture information conversion device 100, the compressed picture information (bitstream) is VBP, it is not possible to allocate proper command bitrate to each picture because bitrate variations from GOP to GOP leads to inappropriate bit allocation.
Also, in the conventional picture information conversion device 100, there is employed an FF buffer for encoding a GOP at the outset in order to calculate the complexity of each GOP picture. So, if the GOP is made up of a large number of pictures, the recording capacity needs to be increased correspondingly, thus elevating the FF buffer cost. In addition, in the conventional picture information conversion device 100, since the structure of the high bitrate compressed picture information, actually input from outside, cannot be recognized until it is entered actually, the optimum recording capacity of the FF buffer is difficult to set. It is also necessary with the conventional picture information conversion device 100 to use the FF buffer if the input compressed picture information (bitstream) is a VBR.
In addition, in the conventional picture information conversion device 100, underflow occurs extremely readily. It is noted that header data including the motion vector (MV) and the control information is encapsulated into the compressed picture information (bitstream) input to this conventional picture information conversion device 100. For these header data, the compressed picture information (bitstream) initially input to the conventional picture information conversion device 100, referred to below as the original compressed picture information (bitstream), is not so crucial because this original compressed picture information (bitstream) is a high bitrate encoded version of the DCT data which is generally more crucial to the header data.
However, if, in the conventional picture information conversion device 100, the compressed picture information (bitstream) is to be compressed further, the DCT data in the compressed picture information (bitstream) is diminished in a larger quantity because in general the header data contents include the control information and hence are not to be changed.
On the other hand, if the compressed picture information (bitstream) output by the conventional picture information conversion device 100 is of the low bitrate, underflow is likely to occur repeatedly s a result of failure in the header data compressing operation. Should underflow occur in the conventional picture information conversion device 100, a pre-set portion of a group of pictures or a sole picture is skipped (dropped). However, the processing of skipping or dropping a pre-set portion of a group of pictures or a sole picture deteriorates the picture quality.
Moreover, in the conventional picture information conversion device 100, the algorithm of the TM5 cannot be immediately applied for detecting the quantization scale because the difference in the pixel intensity region needs to be computed in the TM5.
In the conventional picture information conversion algorithm, calculations of the difference in the intensity region are not taken into consideration. So, with the conventional picture information conversion device 100 employing the conventional picture information conversion algorithm, the cost of the overall device is increased. On the other hand, calculations of the difference in the intensity region are carried out only for I-pictures in the DCT region. If these calculations need to be carried out, the B and P pictures need to be converted into I-pictures. In addition, in certain picture information conversion algorithms, the quantization scale, encoded previously, is used in place of calculating the activity.
Also assume that the conventional picture information conversion device 100 includes a VBV buffer fed with the high bitrate compressed picture information (bitstream). In this case, since the conventional picture information conversion device 100 is unable to comprehend the structure of the GOP included in the high bitrate compressed picture information (bitstream) until the compressed picture information (bitstream) is actually input, it is not known what should be the recording capacity the VBV buffer which is to be in use.
If the VBV buffer of a large capacity is used, the overall cost of the conventional picture information conversion device 100 is increased. Also, if the VBV buffer is unable to cope with the bit allocation processing, the conventional picture information conversion device 100 is readily in trouble due to variations in the input VBR bitstream or in the bitrate.
It is therefore an object of the present invention to provide a method and apparatus for converting the picture information in which, even if the compressed picture information with the variable information volume or with the unidentified information volume is input, the compression processing for the compressed picture information, such as allocating a proper command bitrate to each picture of the compressed picture information, is performed to prevent underflow and/or overflow to suppress deterioration of the picture quality.
In one aspect, the present invention provides a picture information conversion device which is fed with the first compressed picture information of CBR (constant bitrate) or VBR (variable bitrate) as a first bitrate and which outputs the second compressed picture information at CBR as a second bitrate, the first compressed picture information being picture signals compression-coded in terms of a GOP (group of pictures) made up of at least one picture, as a unit, wherein the picture information conversion device includes first storage means for storing N frames of the input first compressed picture information, compressed picture information analysis means for analyzing the structure of the picture type of the GOP of the first compressed picture information input to the first storage means, second storage means for storing the analysis result information as the result of analysis by the compressed picture information analysis means, supplied by the compressed picture information analysis means, pseudo GOP generating means for generating a pseudo GOP, based on the N frames stored in the first storage means and on the analysis result information stored in the second storage means, and code volume allocation means for allocating a pre-set code volume to each picture of the pseudo GOP generated by the pseudo GOP generating means so that the output second compressed picture information will be at the second bitrate. The pseudo GOP predicts the picture type of the entire GOP including the N frames of the first compressed picture information,
In this picture information conversion device, the pseudo GOP generating means generates a pseudo GOP, predicting the structure of the picture types of the overall GOP including N frames of the first compressed picture information, based on the N frames stored in the first storage means and on the analysis result information stored in the second storage means. The code volume allocating means allocates a pre-set code volume to each picture of the pseudo GOP, generated by the pseudo GOP generating means, so that the output second compressed picture information will e at a second bitrate.
In another aspect, the present invention provides a picture information conversion device which is fed with the first compressed picture information of a CBR (constant bitrate) or a VBR (variable bit rate), as a first bitrate, and which outputs the second compressed picture information of a CBR, as a second bitrate, with the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the picture information conversion device includes compressed picture information analysis means for analyzing the first compressed picture information input, complexity calculating means for calculating the complexity of respective pictures of the first compressed picture information input, based on the analysis result information representing the result of analysis by the compressed picture information analysis means, command code volume calculating means for calculating the command code volume allocated to respective picturers of the input first compressed picture information, based on the complexity of the respective pictures calculated by the complexity calculating means and command code volume allocating means for allocating the command code volume calculated by the command code volume calculating means to the respective pictures of the input first compressed picture information.
In this picture information conversion device, the complexity calculating means calculates the complexity of respective pictures of the input first compressed picture information, while the command code volume calculating means calculates the command code volume allocated to the respective pictures of the input first compressed picture information, based on the complexity of the respective pictures as calculated by the complexity calculating means. The command code volume allocating means allocates the command code volume, calculated by the command code volume calculating means, to the respective pictures of the input first compressed picture information.
In still another aspect, the present invention provides a picture information conversion device which is fed with the first compressed picture information of a CBR (constant bitrate) or a VBR (variable bit rate) as a first bitrate and which outputs the second compressed picture information of a CBR as a second bitrate, with the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the picture information conversion device includes data separating means for decoding the variable length code of the input first compressed picture information to separate DCT (discrete cosine transform) data and header data from each other, compressed picture information analysis means for analyzing the input first compressed picture information, and header data compressing means for compressing header data separated by the data separating means if, based on the information pertinent to the command code volume allocated to the respective pictures of the input first compressed picture information, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures.
In this picture information conversion device, the data separating means decodes the variable length codes of the input first compressed picture information and separates the information into DCT (discrete cosine transform) data and header data. The header data compressing means compresses the header data separated by the data separating means, based on the information pertinent to the command code volume allocated to the respective pictures of the input first compressed picture information, in case the command code volume allocated o the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures.
In still another aspect, the present invention provides a picture information conversion device which is fed with the first compressed picture information of a CBR (constant bitrate) or a VBR (variable bit rate) as a first bitrate and which outputs the second compressed picture information at a CBR as a second bitrate, with the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the picture information conversion device includes compressed picture information analysis means for analyzing the input first compressed picture information, complexity calculating means for calculating the complexity of respective pictures of the input first compressed picture information, based on the analysis result information representing the result of analysis by the compressed picture information analysis means, command code volume calculating means for calculating the command code volume allocated to respective pictures of the input first compressed picture information, based on the complexity of the respective pictures calculated by the complexity calculating means, reference quantization scale generating means for generating the reference quantization scale based on the information pertinent to the command code volume calculated by the command code volume calculating means and on the information pertinent to the code volume of the second compressed picture information to be output, activity calculating means for calculating the activity of the input first compressed picture information, quantization scale generating means for generating the quantization scale based on the reference quantization scale generated by the reference quantization scale generating means and on the activity calculated by the activity calculating means, dequantization means for dequantizing orthogonal transform coefficients of the input first compressed picture information and quantizing means for requantizing the orthogonal transform coefficients of the first compressed picture information dequantized by the dequantization means, based on the quantization scale generated by the quantization scale generating means.
In this picture information conversion device, the reference quantization scale generating means generates the reference quantization scale based on the information pertinent to the command code volume calculated by the command code volume calculating means and on the information pertinent to the code volume of the output second compressed picture information. The activity calculating means calculates the activity of the input first compressed picture information, while the quantization scale generating means generates the quantization scale based on the reference quantization scale generated by the reference quantization scale generating means and on the activity generated by the activity calculating means.
In still another aspect, the present invention provides a picture information conversion device which is fed with the first compressed picture information of CBR (constant bitrate) or VBR (variable bitrate) as a first bitrate and which outputs the second compressed picture information at CBR as a second bitrate, the first compressed picture information being picture signals compression-coded in terms of a GOP (group of pictures) made up of at least one picture, as a unit, wherein the picture information conversion device includes first storage means for storing N frames of the input first compressed picture information, compressed picture information analysis means for analyzing the structure of the picture type of the GOP of the first compressed picture information input to the first storage means, second storage means for storing the analysis result information as the result of analysis by the compressed picture information analysis means, supplied by the compressed picture information analysis means, pseudo GOP generating means for generating a pseudo GOP, based on the N frames stored in the first storage means and on the analysis result information stored in the second storage means, the pseudo GOP predicting the picture type of the entire GOP including the N frames of the first compressed picture information, complexity calculating means for calculating the complexity of respective pictures of the input first compressed picture information, based on the analysis result information representing the result of analysis by the compressed picture information analysis means, command code volume calculating means for calculating the command code volume allocated to respective pictures of the pseudo GOP generated by the pseudo GOP generating means, based on the complexity of the respective pictures calculated by the complexity calculating means, command code volume allocating means for allocating the command code volume calculated by the command code volume calculating means to respective pictures of the pseudo GOP, data separating means for decoding the variable codes of the input first compressed picture information for separating the information into DCT (discrete cosine transform) data and header data, header data compressing means for compressing header data separated by the data separating means if, based on the information pertinent to the command code volume allocated to the respective pictures of the pseudo GOP, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures, reference quantization scale generating means for generating the reference quantization scale based on the information pertinent to the command code volume calculated by the command code volume calculating means and on the information pertinent to the code volume of the second compressed picture information to be output, activity calculating means for calculating the activity of the input first compressed picture information, quantization scale generating means for generating the quantization scale based on the reference quantization scale generated by the reference quantization scale generating means and on the activity calculated by the activity calculating means, dequantization means for dequantizing orthogonal transform coefficients of the input first compressed picture information, and quantizing means for re-quantizing the orthogonal transform coefficients of the first compressed picture information dequantized by the dequantization means based on the quantization scale generated by the quantization scale generating means.
In this picture information conversion device, the pseudo GOP generating means generates a pseudo GOP based on N frames stored in the first storage means and on the analysis result information stored in the second storage means. The command code volume allocating means allocates the command code volume, calculated by the command code volume calculating means, to the respective pictures of the pseudo GOP. The header data compressing means compresses the header data separated by the data separating means if, based on the information pertinent to the command code volume allocated to the respective pictures of the pseudo GOP, the command code volume allocated to the respective pictures of the pseudo GOP is smaller than the command code volume of the header data allocated to the respective pictures. The quantization scale generating means then generates the quantization scale based on the reference quantization scale and the activity. Finally, the quantization means re-quantizes the orthogonal transform coefficients of the first compressed picture information dequantized by the dequantizing means, based on the quantization scale.
In still another aspect, the present invention provides a picture information conversion method in which the first compressed picture information of CBR (constant bitrate) or VBR (variable bitrate) as a first bitrate is input and in which the second compressed picture information at CBR as a second bitrate is output, the first compressed picture information being picture signals compression-coded in terms of a GOP (group of pictures) made up of at least one picture, as a unit, wherein the picture information conversion method including the steps of storing N frames of the input first compressed picture information, analyzing the structure of the picture type of the GOP of the first compressed picture information, storing the analysis result information as the result of analysis, generating a pseudo GOP, based on the N frames stored and on the analysis result information stored, the pseudo GOP predicting the picture type of the entire GOP including the N frames of the first compressed picture information and allocating a pre-set code volume to each picture of the pseudo GOP generated so that the output second compressed picture information will be at the second bitrate.
In this picture information conversion method, a pseudo GOP predicting the structure of the picture type of the entire GOP including N frames of the first compressed picture information is generated, based on the stored N frames and on the analysis result information, and a pre-set code volume is allocated to the respective pictures of the generated pseudo GOP so that the output second compressed picture information will be at a second bitrate.
In still another aspect, the present invention provides a picture information conversion method in which the first compressed picture information a CBR (constant bitrate) or a VBR (variable bit rate) as a first bitrate is input and in which the second compressed picture information of a CBR as a second bitrate is output, the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the method includes the steps of analyzing the first compressed picture information input, calculating the complexity of respective pictures of the first compressed picture information input, based on the analysis result information representing the result of analysis, calculating the command code volume allocated to respective pictures of the input first compressed picture information, based on the calculated complexity of the respective pictures and allocating the calculated command code volume to the respective pictures of the input first compressed picture information.
In this picture information conversion method, the complexity of the respective pictures of the input first compressed picture information is calculated, based on the analysis result information, while the command code volume allocated to the respective pictures of the input first compressed picture information, based on the calculated picture complexity. The command code volume, so calculated, is allocated to the respective pictures of the input first compressed picture information.
In still another aspect, the present invention provides a picture information conversion method in which the first compressed picture information of a CBR (constant bitrate) or a VBR (variable bit rate) as a first bitrate is input and in which the second compressed picture information of a CBR, as a second bitrate, is output, the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the picture information conversion method includes the steps of decoding the variable length code of the input first compressed picture information to separate DCT (discrete cosine transform) data and header data from each other, analyzing the input first compressed picture information and compressing the header data separated by the data separating step if, based on the information pertinent to the command code volume allocated to the respective pictures of the input first compressed picture information, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures.
In this picture information conversion method, the variable length codes of the input first compressed picture information are decoded and separated into DCT (discrete cosine transform) data and header data, and the separated header data is compressed if, based on the information pertinent to the command code volume allocated to the respective pictures of the input first compressed picture information, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures.
In still another aspect, the present invention provides a picture information conversion method which is fed with the first compressed picture information of a CBR (constant bitrate) or a VBR (variable bit rate) as a first bitrate and which outputs the second compressed picture information at a CBR as a second bitrate, the first compressed picture information being picture signals compression-coded in terms of a picture as a unit, wherein the picture information conversion method includes the steps of analyzing the input first compressed picture information, calculating the complexity of respective pictures of the input first compressed picture information, based on the analysis result information representing the result of analysis, calculating the command code volume allocated to respective pictures of the input first compressed picture information, based on the calculated complexity of the respective pictures, generating the reference quantization scale based on the information pertinent to the calculated command code volume and on the information pertinent to the code volume of the second compressed picture information to be output, calculating the activity of the input first compressed picture information, generating the quantization scale based on the reference quantization scale generated and on the activity calculated, dequantizing orthogonal transform coefficients of the input first compressed picture information, and re-quantizing the dequantized orthogonal transform coefficients of the first compressed picture information, based on the generated quantization scale.
In the present picture information conversion method, the reference quantization scale is generated, based on the information pertinent to the command code volume and to the information pertinent to the code volume of the output second compressed picture information, to calculate the activity of the input first compressed picture information. The quantization scale is generated, based on the generated quantization scale and on the calculated activity. The orthogonal transform coefficients of the dequantized first compressed picture information are re-quantized based on the generated quantization scale.
In yet another aspect, the present invention provides a picture information conversion method in which the first compressed picture information of CBR (constant bitrate) or VBR (variable bitrate) as a first bitrate is input and the second compressed picture information at CBR as a second bitrate is output, the first compressed picture information being picture signals compression-coded in terms of a GOP (group of pictures) made up of at least one picture, as a unit, wherein the picture information conversion method includes the steps of storing N frames of the input first compressed picture information, analyzing the structure of the picture type of the GOP of the input first compressed picture information, storing the analysis result information as the result of analysis, generating a pseudo GOP, based on the stored N frames and on the stored analysis result information, with the pseudo GOP predicting the picture type of the entire GOP including the N frames of the first compressed picture information, calculating the complexity of respective pictures of the input first compressed picture information, based on the analysis result information representing the result of analysis, calculating the command code volume allocated to the respective pictures, based on the complexity of the respective pictures calculated by the complexity calculating step, allocating the calculated command code volume to respective pictures of the pseudo GOP, decoding the variable codes of the input first compressed picture information for separating the information into DCT (discrete cosine transform) data and header data, compressing separated header data if, based on the information pertinent to the command code volume allocated to the respective pictures of the pseudo GOP, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures, generating the reference quantization scale based on the information pertinent to the command code volume calculated and on the information pertinent to the code volume of the second compressed picture information to be output, calculating the activity of the input first compressed picture information, generating the quantization scale based on the generated reference quantization scale and on the activity calculated, dequantizing orthogonal transform coefficients of the input first compressed picture information and re-quantizing the dequantized orthogonal transform coefficients of the first compressed picture information based on the generated quantization scale.
In the present picture information conversion method, the pseudo GOP is generated, based on the stored N frames and on the analysis result information, and allocates the calculated command code volume to the respective pictures of the pseudo GOP. If, based on the information pertinent to the command code volume allocated to the respective pictures, the command code volume allocated to the respective pictures is smaller than the command code volume of the header data allocated to the respective pictures, the separated header data is compressed. The reference quantization scale is generated, based on the information pertinent to the calculated command code volume and on the output second compressed picture information, and the activity of the input first compressed picture information is calculated. The quantization scale is generated based on the so-generated reference quantization scale and on the calculated activity. Finally, the orthogonal transform coefficients of the dequantized first compressed picture information is re-quantized. With the picture information conversion method and device according to the present invention, if the compressed picture information with a variable or non-recognized information volume is input, optimum compression processing on the compressed picture information, such as allocation of an optimum command code volume to respective pictures of the compressed picture information, is performed to prevent underflow and/or overflow to suppress deterioration of the picture quality.